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ABS TRACT 


The possibility of designing constrained adaptive finite 
impulse response digital filters is investigated as motivated 
by a study of adaptive noise cancellation. The first con- 
straint considered consists of a fixed angle between filter 
zeros and is implemented in a master-slave approach in which 
one of the zeros is adjusted adaptively and the others follow 
subject to the constraint. The second constraint considered 
is a linear constraint on the filter weights and is imple- 
mented by augmenting the error equation with Lagrangean 
multipliers. Simulations indicate that the approach is 


feasible. 
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ieee N LD RODUCTION 


This thesis investigates the possibility of designing 
constrained adaptive finite impulse response (FIR) filters. 
FIR adaptive filters self-adjust their parameters to match 
the output signal to a desired signal in an optimal least 
Squares sense. Considered in detail is the possibility of 
designing the adaptive process so that either the parameters 
(weights) of the filter are constrained by an algebraic for- 
mula or, alternatively, are constrained so that the zero 
pattern of the FIR transfer function is fixed while the 
actual positions of the zeros are adaptively adjusted. An 
application which motivated this study is adaptive noise 
cancellation where noise rejection over a given bandwidth 
with a specified frequency rejection spectrum is required, 
but the center frequency of the rejection band is unknown and 
needs to be determined adaptively. 

The usual method of extracting a signal from a strong 
noise background is to pass it through a filter which tends 
to suppress the noise while leaving the signal relatively 
unchanged. Filters designed for this purpose can be either 
fixed or adaptive. The design of fixed filters is based on 
some prior knowledge of both the signal and noise character- 
istics. Adaptive filters have the distinct advantage of 
being able to adjust themselves automatically and their 


implementation requires little a priori knowledge of the 





Signal and noise characteristics. This type of adaptive 
filtering often converges to the optimal filter, which 
Originated with the pioneering work of Wiener [Ref. 1] and 
later was enhanced by Kalman [Ref. 2] and others. The 
optimal Wiener filter is defined as the linear filter opti-~ 
mized with respect to a minimum mean squared error, where the 
error is the difference between the filter output and the 
desired filter output. 

Widrow [Ref. 3] presents the classic FIR adaptive filter 
which is optimized via a gradient minimum seeking algorithm 
called the Least Mean Squared (LMS) algorithm. This chapter 
discusses the theory behind both the Wiener and LMS filters, 
and demonstrates that, for statistically stationary input 
Signals, the steady-state values of the LMS adaptive filter 
weights are accurate approximations of the Wiener weights. 

The following matrix, vector, and scalar definitions are 


used in this thesis. An underlined capital letter denotes a 


Matrix (M) . A lower case underlined letter denotes a vector 
(Vv). A lower case letter which is not underlined denotes a 
scalar (s). Finally, a capital letter which is not under- 


lined denotes an internal element of the corresponding matrix 
Min) . 

In Chapter II the concept of adaptive noise cancelling is 
studied in detail. In Chapter III the constrained adaptive 
FIR filter is presented and in Chapter IV some simulation 


results are given. 





A literature search has indicated that very little re- 
search has been done in the area of constrained adaptive 
filters. Frost [{Ref. 8] presents a constrained LMS algorithm 
which is capable of adjusting an array of sensors in real 
time to respond to a signal coming from a desired direction 
while discriminating against noises coming from other direc- 
tions. A set of linear constraints on the weights maintains 
a chosen frequency characteristic for the array in the 


direction of interest. 


A. THE ADAPTIVE LMS AND THE OPTIMAL WIENER FILTERS 

The LMS adaptive filter shown in Figure 1.l uses the 
weighted sum of a set of input signals which are combined to 
form an output signal y(n). The input signal vector is 


defined as 


_ T 
Nae nme (nN). XC] 
Lia 
= [(x(n) X(n-1) ... xX (n-n)]? 
The weighting coefficients 
w= (W. Wi, ... WI) (92) 


are the weights of the system. Each input value is multi- 
plied by a corresponding weight coefficient and the linear 
combination of the sum of these weighted inputs forms the 


output 





x I(i) 






x2 (i) 
y (J) 


FILTER 


x 5()) OUTPUT 


INPUT SIGNALS 


d(}) 


DESIRED 
REFERENCE 


x n{j) 


Fig. 1.1. The LMS Adaptive Filter 
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N N 
ike) as eae = ) W. X(n-i) (ies) 


Mamehn can be written in matrix form as 


y(n) = X(n)"W = WX(n) (1.4) 


An error Signal is now defined as the difference between 


a desired response d(n) and the actual response y(n). 


e(n) = d(n) - y(n) = d(n) - W'X(n) (1.5) 
The purpose of the adaptive algorithm is to adjust the 
weights of the filter to minimize the mean-square error. A 
general expression for mean-square error as a function of the 
weight values, assuming that the input signals and the de- 
Sired response are statistically stationary, can be derived 


in the following manner. Squaring the error results in 


e(n)* = d(n)? = 2d(n)x(n) Tw +w? x(n) x(n) 7W (1.6) 


Taking the expected value of both sides yields 


2 T 


E{e(n)*] = E{d(n)*J - 2E{d(n)X(n)7]w 


ab 


+ WE[X(n)X(n) 7] (1.7) 


If the vector Cd is now defined as the cross correlation 
between the desired response (a scalar value) and the input 


vector, the result is 


JUaE 





rq = Eld(n)X()] = Eld(n)X(n) d(m)X(m-l) ... d(n)X(n-N)] (1.8) 


The input correlation matrix Ree is defined as 


Pannen) = ema XK (ma) ee | OX (nn) XN (nm) 
Pen) cme xe im) XZ (ni) Ss SCC XZ (rn) XN (rn) 
* E[X(n)X(n)"] = E 
XN(n)X1(n) XN(n)X2(n) ... XN(n) XN(n) 
(1.9) 
X(n)* Min knees | Xn) X (nN) 
ba Sie) oe Gpea vee X(n=1)X(n-N) 
= E 
X(n-N)X(n) X(n-N)X(N-L) ... Fahey 
(ie-1:0,) 
Equation (1.10) is usually written as: 
Ry (9) Ry (4) re Ry (N) 
Rly 61) Ryy 69) gas Ry (N-1L) 
R = . (1.11) 
—XX 
Ry (N) Ri (N-L) ee a) 


Now the mean squared error (1.7) can be expressed as 
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i. a A T 
Ele(n)“] = Eld(n)] - 2x qi + WR W (eee) 


Since the error is a quadratic function of the weights 
and Rix is a positive definite matrix then the surface of the 
error has a guaranteed global minimum. Gradient methods 
adjust the weights to minimize the error by descending along 
this surface with the objective of finding the bottom. 

mie Gradient V of the quadratic error function is obtained 
by differentiating Equation (1.12) with respect to the weight 


vector w. 


EVe Cn) 3E{e(n)“] Een aie i 
” | aaa am ae “Ce } (Glee?) 
I I . N 
y = ~2 Ed + AR ire 1 43 


This optimal weight vector w is called the Wiener weight 
vector [Ref. 1] and is found by setting the gradient of the 
mean square error function to zero. 


we = R? 


This equation is a matrix version of the Wiener-Hopf 
equation. The practical objective of the adaptive system is 
Bemeting a solution to (1.15). An exact solution would require 
a priori knowledge of the correlation matrices r., and R,,. 
However, this information is usually not Seen, Addi- 


tionally, when the number of weights is large a direct 


he 





solution is computationally cumbersome since it requires an 
N by N matrix inversion in addition to the N(n+l)/2 auto- 


correlation and cross correlation measurements. 


B. AN EXAMPLE OF THE WIENER SOLUTION 

As an example to illustrate the calculations involved 
for a simple four weight Wiener solution consider the 
following example using deterministic signals. The input 
Signal is the sampled sum of two sinusoids of different 


frequencies and is given by 


X(nT) = sin (w nT) + sin(wi~nT) (eG) 


2 


The desired value is the sampled desired or reference 


Signal. 


d(nT) = K sin(w.~anT) (a7) 


2 


In order to calculate the autocorrelation matrix, Ros! note 


that 
Bi(n) = X(n) = sin (wnt) + Sin (w nT) 
X2(n) = X(n-l1) = sin(w, (n-1)T) + sin(w, (n-l)T) 
(ie) 
Mam) = XX{na—2Z2) = sin(w, (n-2)T) + Sin(w, (n-2)T) 
X4(n) = X(n=-3) = Sin (w, (n-3)T) + sin (w, (n-3)T) 
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The following expected value computations are taken over a 
full number of cycles, P, for both sinusoids. 


2 


E[X1(n)X1(n)] sili) 2 saiene 


Likewise all the diagonal terms will take on the value of 
K2(n)? = x3(n)? = x4(n)* = 1 


Consider now the off diagonal terms of Rox 


Pee (n) X2(n) | Kita) X2°(n) 


a 
) (sin wjnT+sin w nT) (sinw, (n-1)T +sinw, (n-1)T) 
n=0 


Carrying out the indicated multiplication yields 


Pp 
E [X1(n) X2 (n) ] = ) (sinw, nT) (sinw, (n-1)T) 
n=0 
Pp 
+ ) (sinw nT) (sinw, (n-1) T) 
n=0 
p 
+ ) (sinw,nT) (sinw, (n-1)T) 
n=0 
Pp 
+ ep 2? (etna, (n-1)) 


1G 





The final two terms of this expression are zero leaving 


Pp 
E([Xl(n)X2(n)] = p (sinw nTsin(w)nT-w,T)+sinw nTsin(w,nT-w,T) ) 


Using the identity (sinA) (sinB) = =[cos (AB) -cos (A+B) } the 


final expression becomes 


E([X1(n)X2(n)] = 5 cosw,T + > cosu,T (1.19) 
Similarly, 
E([X1(n)X3(n)] = 5 cos2w,T + F cos2w,T (1.20) 
E(X1(n)X4(n)] = 5 cos3u,T + > cos3u,7 aie 
Denoting 
E(Xl(n)X2(n)] = a 
Fixe (n) x3) =~ b (22) 
Mieka) x4kn)j° = ¢ 


the autocorrelation matrix can be expressed as 


= (223) 





For the cross correlation matrix rg 


= = Pcimy <L(n) a(n) xX2(n) d(n) X3(n) d(n)x4(n)]? Cie Z4) 


These terms compute as 


fein) X1(n) J K(sinw, nT +sinwjnT) sinw, nT 


ll 
A 
M1 


Sinw nT (sinw nT +sinwnT) 


2 2 P 
= K ) sin°w.nT + K )}) (sino 


ne) sinwg.n Td 
a i 
n=0 


Ht 2 


The final term sums to zero leaving 
2 K 
E[a(n)Xl(n)] = K } sin°wnT = 5 G25) 


Similarly, 


Ppa (n) X2(n) | K(sinw, (n-1)T +sinw, (n-1)T)sinw, nT 


Pp 
= kK at, sinw,nT(sinu, (n-1)T) 


P 
aK .¥ sinw,nT(sinw, (n-1)T) 


The final term sums to zero leaving 
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Ie le 
E{d(n)X2(n)] = KI } afin ey ec ie) Po dime 1) 
ad i i iL 1 
n=0 n=0 
Using the (sinA) (sinB) trigonometric identity invoked for 


(1.19) and realizing that the final term sums to zero, the 


expected value finally becomes 


E{d(n)X2(n)] = 5 cosw,T (1.26) 
Likewise, 

E{d(n)X3(n)] = $= cos2w,T (la) 
and 

E[d(n)X4(n)] = = cos} T GL eh 


The Wiener solution for w* as given by (1.15) is 


1 
al a b o a 
a 1 a b cosw,T 
We = : > (1.29) 
b a il a cos2u,T 
S b a 1 cos3u,T 
where the values of a, b and c are given by (1.22). The 


Wiener vector is calculated for the specific values of 
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—e2 = 35 Hz 
t= 1/256 


For these specific values 


a = .8116 

b= .3676 

ee —~.052 

K = 
xlcosuT] = .485 
K 

s(coswT] = .441 
2 

K 

zl coswuT] =e 


and the optimal Wiener vector w* is 


wi 3.0614 
we -5.5772 
w* = = CEs 5) 
we Sy Les 
Wi -1.5780 


J 


C. THE LMS ADAPTIVE FIR FILTER 

The values for the correlation matrices are not generally 
known a priori. The LMS adaptive algorithm introduced by 
Widrow and Hoff (Ref. 1] is a practical method for finding 
close approximate solutions to (1.15) in real time. The 
algorithm does not require measurements of correlation 
functions, nor does it require matrix inversion. The LMS 


algorithm is an implementation of the method of steepest 


Ie, 





descent. According to this method, the "next" weight w(n+l) 
1s equal to the present weight w(n) plus a change proportional 


to the negative gradient. Thus 


w(ntl) = w(n) - ypV(n) 


The parameter yu controls stability and also the rate of 
convergence. An estimate of the instantaneous gradient V(n) 
is obtained by assuming that the square of a single error 
sample «(n) is an estimate of the mean square error. Differ- 


entiating <«(n) with respect to w results in 





ye 2 2 2T 

ain) | (= (2ginl” detn) _ detn) gece) 
1 2 n 

- 7 d3e(n) 3e(n) je(n),2 

y(n) = 2 e(n) ow, BW ee ow J (13233) 


The expression for the gradient estimate can be approximated 


by 


Tne =] = 2e (n) X {(n) (1.34) 


Using this estimate in place of the true gradient yields the 


Widrow-Hoff LMS algorithm given by 


w(ntl) = w(n) + 2ye(n)X(n) G4-73.5)) 


The algorithm is generally easy to implement and al- 


though it makes use of gradients of the mean square error 
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function, it does not require squaring, averaging or 
differentiation. 
To show the convergence of the Widrow-Hoff LMS algorithm 


(1.35) to the Wiener solution given by (1.15) write 


w(ntl) = win) + 2ue(n)X(n) 

as 
wintl) = win) + 2uX(n) [d(n) -X(n)“w(n) ] 
w(ntl) = [I -2uX(n)X(n)*]w(n) + 2uX(n)d(n) 


Consider now the ensemble average. That is, 


E[w(n+l)] = [I ~ 2uR, J Elwin) | + Aurea (als 6) 
Meeieam Initial weight vector w(0), j+l iterations of 
Equation (1.36) becomes 

| _ _ j+1 ; _ i 
E{w(jt+l)] = [I 2uR,.! w (0) t2u. ai 2uR,,! rxd ae oe) 


closelal nn) oe — 


This equation may be put in diagonal form by using the normal 


form expansion of the matrix Roy! that is 


Za Ak 





where A is the diagonal matrix of eigenvalues, and the square 


Matrix of eigenvectors is the matrix Q. Equation (1.37) 


can now be written as 


[lw(j+l) } [I-2uQ"*aQ] 3*1,(0) + 2u } [I-20 *aQ] “Bas 


i=0 


As long as the terms of the diagonal matrix [I-2uA] are all 
of magnitude less than unity, then the first term of (1.39) 


goes to zero as the number of iterations increases. That is, 


Taian ie—Aeni] Dee 


j72 


Writing (1.40) as a geometric series results in 


lim ; [I-2yuA] = 
Binaas i=0 


or for the specific component 'p' of the matrix 


1 
Teh 
70 





Lim } [I-2uA]~ = 
jr 1= © 


Therefore, in the limit, Equation (1.39) becomes 


lim Etw(j+l)] = Q “AQr,, = R 


j70 


Da J 


of ir-2uarS*owio) +2u ) [I-2uQ"~AQ]*z, (1. 
i=0 z 


+ 0 (be 


39) 


40) 


41) 


(42) 





which is the same as the Wiener solution shown in Equation 
eel). 

Convergence of the mean of the weight vector to the 
Wiener solution is insured if and only if the proportionality 
constant yp is set within certain bounds. Since the diagonal 
terms of [I-2uA] must all have magnitude less than unity, 
and since all eigenvalues in A are positive, the bounds on u 


are given by 


Be a 


Queer yr <1 KX (1.43) 


where i is the maximum eigenvalue of R_.. 
max —XX 
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It. ADAPTIVE NOISE CANCELLING AND ITS APPLICATIONS 


In this chapter the concept of adaptive noise cancella- 
tion is considered. Adaptive noise cancelling is one of the 
most practical applications of adaptive signal processing 
[Refs. 4,5,6,7]. The basic principle involved is the use of 
a reference input derived from one or more sensors located 
at points in the noise field where the signal is either unde- 
tectable or weak. The reference input is adaptively filtered 
and subtracted from the primary input containing both signal 
and noise to generate an error signal which controls the 
adaptive process. The result is the attenuation or elimina- 
tion of the primary noise by cancellation. In circumstances 
where adaptive noise cancelling is applicable levels of noise 
rejection are often attainable that would be difficult or 
impossible to achieve through direct filtering. Because the 
concepts of adaptive noise cancelling and their applications 
are fundamental to constrained adaptive filtering developed 
in Chapter III, they are presented here in detail. For exam- 
ple, if the angle between zeros and the magnitude of the zeros 
are maintained constant during an adaptive process, the fre- 
quency response characteristics remains invariant. Specific 
examples are discussed and the resulting equations indicate 


the design limitations of these approaches. 


A. THE BASIC SYSTEM 
Figure 2.1 depicts the basic adaptive noise cancelling 


system concept . A signal [s] is transmitted over a channel 
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ISYSTEM 
SIGNAL z= S+N, OUTPUT 
SOURCE A 


| 

| 

| 

| 

) FILTER 

IKON OUTPUT 

NOISE °! ADAPTIVE 
SOURCE FILTER 


Fig. 2.1. The Adaptive Noise Cancelling Concept 
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to a sensor that picks up a noise [nO] which us uncorrelated 
with the signal. The combined signal and noise [s+n0] form 
the primary input to the canceller. A second sensor re- 
ceives a noise [nl] which is also uncorrelated with the 
Signal but which is correlated in some unknown manner with 
the noise [n0]. This sensor input provides the reference 
neo tC to the canceller. The noise [nl] is filtered to pro— 
duce an output [y] that is as close a replica as possible 

to [nO]. This output is subtracted from the primary input 
[stn0] to produce the system output [z = s+n0-y]. 

If it were possible to know the characteristics of the 
Channels over which the noise was transmitted to the primary 
and reference sensors, then it would theoretically be possi- 
ble to design a fixed filter capable of changing [nl] into 
[nO]. The filter output could then be merely subtracted from 
the primary input to produce the signal alone. However, the 
Characteristics of the transmission paths are generally un- 
known or known only approximately and are seldom of a fixed 
nature, therefore precluding the use of a fixed filter. 

In the system shown in Figure 2.1 the reference input is 
processed by an adaptive filter. Self-adjustment of the 
filter's impulse response is accomplished by the LMS algorithm 
described in Chapter I. The error signal used in the adaptive 
process depends on the nature of the application. With noise 
cancelling systems the practical objective is to produce a 
system output [z = s+n0O-y] that is a best fit in the least 


Squares sense to the signal [s]. 


iG 





Consider the system inputs shown in Figure 2.1 and the 
filter output [y]. Assume that [s] is uncorrelated with [n0] 
and [nl] and that [n0] and [nl] are correlated. The output 


fz] is 
Z = stnd-y eZine) 
Squaring produces 
2 


Z = 5° + (noe) + 2s (n0-y) ( 22927) 


Now, taking the expectation of both sides and using the fact 


that [s] is uncorrelated with [n0] and [y] produces 


E[2°] E[s*] J. E{ (n0-y)*] + 2E[s(n0-y) ] 


E[s*] + E[(n0-y) 7] ee) 
The minimum output power is 
2 2 ; 2 
Min E{Z ] = E{s ] + Min E[(n0-y)™] (2.4) 


The Signal power is unaffected as the filter is adjusted to 
minimize E[z°]. When the filter is adjusted so that E[27] 

1S minimized, E[ (n0-y)7] is therefore also minimized. The 
Filter output [y] is then a best least squares estimate of the 
primary noise [nO]. It is also of interest to note that 

when E[ (n0-y) 7] is minimized, a (pee) is also minimized, 


Since from (2.1) 
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(z-s) = (n0-y) (2735) 


Adapting the filter to minimize the total output power causes 
the output [z] to be a best least squares estimate of the 
signal [s] for the given reference input. The output [Zz] 

will contain the signal [s] plus noise. From (2.1) the output 
noise is given by (n0O-y). Since minimizing B(27] minimizes 
Bmno-y)*), minimizing the total output power minimizes the 
Output noise power. Because the output signal remains con- 
stant, minimizing the total output power maximizes the signal- 
to-noise ratio. From (2.3) it can be seen that the smallest 
possible output power E(z7] = E[s*] is achieved when 

=] = 0 and therefore y = nO and z= s. In this case, 
Minimizing output power causes the output signal to be per- 
fectly noise free. These same arguments can be extended to 
the case where the primary and reference inputs contain, in 
addition to [{n0] and [nl], additive random noises uncorre- 
lated with each other and with [s], [n0] and [nl]. They can 
also be extended to the case where [n0] and [nl] are deter- 


Ministic rather than stochastic. 


B. SIGNAL-TO-NOISE RATIO IN THE ADAPTIVE FILTER 

At this point it is of value to show analytically the 
increase in signal-to-noise ratio of the noise cancelling 
technique. 

As noted previously, fixed filters are generally inappro- 


priate for noise cancelling situations because the correlation 
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and cross correlation functions are usually unknown and 
additionally they often vary with time. Adaptive filters 
rlearn” these statistics initially and then track them through 
slow variations. For stationary stochastic inputs, however, 
the steady-state performance of adaptive filters closely 
approximates that of fixed Wiener filters and therefore Wiener 
filter theory provides a useful mathematical tool in care- 
fully analyzing statistical noise cancelling problems. 

Figure 2.2 shows the classic single-input/single-output 
Wiener filter. The input signal is x(j), the output signal 
1s y(j) and the desired response is d(j). The input and 
output signals are assumed to be discrete in time, and the 
input signal and desired response are assumed to be statis- 
tically stationary. The error signal is e(j) = ad(j) - y(j). 
The filter is linear, discrete and designed to be optimal 
in the minimum mean-square-error sense. It is considered 
to be composed of an infinitely long, two sided tapped delay 
aime . 

As shown in Chapter I (1.15) the optimal Wiener weight 


solution can be written 
afi (ie) 4 Bs a) ees (2.6) 
= —xXX == 


or 


WH(2)RL (KL) = Deg (i) (em) 


Zo 








DESIRED RESPONSE 
d(j) 


Fig. 2.2. The Wiener Filter 


30 


“Dear cUmesd! es 





. Fee 


70 


‘iat same. oD yes see 


To obtain the transfer function of the Wiener filter 
consider first the power spectral density of the process. 


The application of the z-transform to Roy (*) yields 


- 7 -k 
Za a Ric) Z (2.8) 


aiiaiel k=-0 
Likewise, the cross power spectrum between the input signal 


and desired response is 


erae= zir tk] = ) x atk)z* (2.9) 


w*(z) ££ J w*(k)z~ (2.10) 
kK=—-0 
For specific values of an individual matrix the optimal 
Wiener transfer function can be written as 


Sq‘) 


w*(z) = = Zs (241) 


Consider now a single channel adaptive noise canceller 
with a typical set of inputs shown in Figure 2.3. The pri- 
Mary input consists of a signal s(j) plus the sum of two 
noises m0(j) and n(j). The reference input consists of a 
sum of two other noises ml(j) and n(j) * h(j), where h(j) is 
the impule response of the reference channel whose transfer 


MmmecctiOn iS H(z). To simplify the notation the transfer 
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function of the noise path from n(j) to the primary input 

has been set at unity. This procedure does not restrict the 
analysis since a suitable choice of H(z) and of statistics 
for n(j) will allow any combination of mutually correlated 
noises to appear at the primary and reference inputs. The 
noises m0(j) and ml(j) are uncorrelated with each other, with 
s(j) and with n(j) and n(j) * h(j). The noises n(j) and 

n(j) * h(j) have a common origin and are correlated with each 
other but uncorrelated with s(j). 

The noise canceller includes an adaptive filter whose 
reference input x(j) is ml(j) + n(j) * h(j3) and whose desired 
response d(j) is the primary input to the noise canceller and 
is composed of s(j) + mO(j) + n(j). The error signal e(j) 
is the noise canceller's output. When the adaptive process 
has converged then the optimal unconstrained transfer function 
of the adaptive filter is given by (2.11), which can be further 
reduced as follows. The spectrum of the noise ml is 


S (z) and that of the noise n arriving via H(z) is 


miml 
s_(z) |H(2) |*. Therefore, the input spectrum to the filter 
is 


_ 2 
S..(z) = § (z) + Sn (2) (H(z) | fo2ee 12) 


XX mimlL 


The cross power spectrum between the filter's input and 
the desired response depends only on the mutually correlated 


primary and reference inputs and 1s given by 
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es aol 
Sig (2) = San $2) Hz ) (2.3) 
Mere cucing Equations (2.12) and (2.13) into (2.11) results 
in the Wiener transfer function and is given by 

S__(z)H(z~) 
w*(z) = <a (2.14) 
S aim (2) + Syn (2) |H(z) | 

The result is a transfer function which is independent of 
the primary Signal spectrum Si, (2) and of the primary uncorre- 


(Zi 


lated noise spectrum Sn0m0 


An interesting special case which clearly brings out the 


function of the adaptive noise canceller is when the additive 


noise ml in the reference input 1S zero. Then Saeed 1s 
zero and the optimal transfer function becomes 
w*(zZ) = 1/H(z) (2.15) 


That is, the adaptive noise canceller causes the n(j) noise 
to be perfectly nulled at the noise canceller output. The 
primary uncorrelated noise m0(j) remains uncancelled. 
Consider an analysis of the performance of the adaptive 
noise canceller in terms of the ratio of the signal power 


density at the output, o (z), to the noise power density at 


Owe 
the primary input Popes, OE) This ratio can be written as 
primary noise 
Pour ‘2? 2 San’?! * Snomo ‘2? _ pwr spectrum Orc) 
Pari (2) Soutput a C4 output noise 


pwr spectrum 
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with the signal power spectrum factored and cancelled out of 
the numerator. Figure 2.3 shows that the output noise 
spectrum consists of the sum of three components--one due to 
the propagation of m0(j) directly to the output, another due 
to the propagation of ml(j) to the output via the transfer 
function W(z), and another due to the propagation of n(j) 

to the output via the transfer function 1 -H(z)W(z). The 
output noise power spectrum can then be written as 


S (my = S somo (2) +Smymy (2) [we lz) |? 


output noise 


+S (z) (1 -H(z)W*(z) |* (2.17) 


Now, if the ratios of the spectra of the uncorrelated noises 
to the spectra of the correlated noises (noise-to-noise den- 
Sity ratios) at the primary and reference inputs are defined 


as 


Snomo '?) 
AS) ee 


nn 


C253) 


S (GZ) 
B(z) = —aiml (2.19) 


s_(z) |H(2) |* 


then the transfer function of (2.14) can be written as 
Wweez) = 1/{H(z) (B(z)+1)] (2520) 


and the output noise power spectrum (2.17) can be written 


as 
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—— 


output noise ‘?? ~ Smomo ‘2? u iH (z) |? B(z)+1)° 
1 
a Sin (2) [1 B(aye1 | 
= S$ (2)a(z) +S (2) i ay 


Sa@emene ratio of the output to the primary input noise power 


spectra given in (2.16) is 


2) San $2) tt + A(z) ] 


Pora ‘2) S eZ) 


output noise 


[1 + A(z) 
AZ )a+ en / 


] 
CBaeZEe ) 


= A(z) + 1)(B(z) +1) CRL2) 
Diz Pi Z)B(zZ) + Biz) 

This expression allows an estimation of the level of noise 
reduction to be expected with an ideal noise cancelling sys- 
tem. In such a system the signal propagates to the output 
feenea transfer function of unity. From (2.22) it can be 
seen that the ability of a noise cancelling system to reduce 
noise is limited by the uncorrelated-to-correlated noise den- 
Sity ratios at the primary and reference inputs. The smaller 
in magnitude are A(z) and B(z), the greater will be 

7 oor 2) and the more effective the action of the 
Gencelier. The desirability of low levels of uncorrelated 


noise in both inputs is made even more evident 
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by considering the approximations 


Pout ‘2? Peele Zz) 
1) small A(z) Pari (2) = — Biz) (27923) 
2) small B(z) Pout (7) 2 1 + A(z) AA 
5 (Zz) SSC) Been! 
pLEt 
Pont 62? = 1 


Infinite improvement is implied by these relationships 
when both A(z) and B(z) are zero resulting in complete re- 
moval of noise at the system output and perfect signal repro- 
duction. When both A(z) and B(z) are small other factors 
such as misadjustment caused by gradient estimation noise in 
the adaptive process and the finite length of the adaptive 
filter limit system performance. These factors are discussed 


at some length in [Ref. 3]. 


C. SIGNAL PROPAGATION IN THE REFERENCE INPUT 

If it is reasonable to consider reference noise propaga- 
tion into the primary signal input it is also reasonable to 
consider certain instances when the signal propagates to the 
reference input. The system depicting the adaptive noise 
canceller with signal components in the reference input is 
Shown in Figure 2.4. The derivation which discusses how much 


of the signal is cancelled when a portion of the signal input 


S 7 





Adaptive Noise Canceller With Signal 
Components in the Reference Input 
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"leaks" into the reference input omits the additive uncorre- 
lated noises m0(j) and ml(j) in order to simplify the 
analysis. 

Given that the spectrum of the signal is So, (2) and 
that of the noise is San $2) 9 then the spectrum of the 


reference input is given by 
ees ay incan i += s_ (z) H(z) | QE) 
ss nn : 


The cross spectrum between the reference and primary inputs 
is 
Sq (2) = S,,(2) Iz ) + Sin (2) H(z ) (227) 


When the adaptive process has converged, the Wiener trans- 


fer function of the adaptive filter given by (2.11) is 


52. (a) Ha te S__(z)H(z~) 


_ S (z) |I(z) |* + S (z) |H(z) |? —— 
ss nn 
The transfer function of the propagation path from the 
Signal input to the noise canceller output is 1-I(z)W*(z) 
and that of the path from the noise input to the canceller 
output is 1-H(z)W*(z). The spectrum of the signal component 


in the output is thus 


- _ 2 
5 Ae S.,(2) | I (z)W* (z) | 
|H(z)-I(z) |S) (2) H(z7>) 2 
~ Sash. 2 2 ae 
S_ (2) |I(z) | +S_ (2) | H(z) | 
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and likewise, that of the noise is 














= Z 
ae, \Z) = S an (2) [2 - H(z)W*(z) | 
[I (z)-H(z) 1S. .(z)1(z>) 2 
= $ (2) ———__$3-______, (2.30) 
S ., (2) |I(z) | +S_ (2) (H(z)) 
The output signal-to-noise density ratio is therefore 
-1 
' (2) : ss San ‘?) BZ a 2 
ou Z - 
nn Ss, (2) MZ.) 
2 
S__ (z) |H(z) | 
= Ln (2-52) 


Z 
5, A) ee G2) 


The output signal-to-noise density ratio can be conveniently 
expressed in terms of the signal-to-noise density ratio at 
the reference input in the following manner. The spectrum 


of the signal component in the reference input is 


_ 2 
Sa ae) ite za (232) 


and that of the noise component is likewise 


7 2 
S (z) = S (2) [H(2) | (2.23) 


nn ref 


Therefore, the signal-to-noise density ratio at the reference 


input is thus 
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CONE aC 0 ie 
Cee 2 (2.34) 
_ S (2) |H(2) | 


Comparison of Equation (2.34) with (2.31) shows that 
(z) (2235) 


This shows that if the noises in the primary and reference 
inputs are mutually correlated, the signal-to-noise density 
ratio at the noise canceller output is simply the reciprocal 
at all frequencies of the signal-to-noise ratio at the reference 
input. That is, in order to obtain a good signal-to-noise 
density ratio at the filter output there should be very little 
Signal at the reference input. 

The final objective of the analysis is to derive an 
expression for the spectrum of the output noise. As with the 
previous analysis it is instructive to first write the trans- 
fer function for the path from which the noise n(j) propagates 


GO the output. 


1 


a es 
S _ (2) Iz )+S_ 2) H(z ) Z 


S 


1-H(z)W*(z) = 1-H(z) —}———,—______,, 
S_, (2) |I(z) |“+s_ (2) [H(z) | 


S _(z)I(z-*) [I(z) -H(z) ] 
ee (2.36) 
S_, (2) |I(z) | +S (2) |H(z) | 


When I(z) is small (2.36) reduces to 
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“S_,(2)1(z >) 


ee) ek) ee — Gee) 
San (2) H(z ) 
The output noise spectrum is 
Z 
= pa * 
S output noise Sin (2) 1 Me) (Z) | (2.38) 


Again, considering the case where I(z) is small results in 


$__(2)Iiz 
I 


Son (Z) H(z ) 


DD 
(Ze So 7) (2.39) 


Soutput noise nn 








If Equation (2.39) is written in terms of the signal-to-noise 
density ratios at the reference and primary inputs, where 
the signal-to-noise density ratio at the reference input is 


given by Equation (2.34) and 


Se Zi 
A ss 
0  .(z2) = =—_- (2.40) 
jonqal Sin {2 
then Equation (2.39) can be written as 
output noise Son (2) lPpeg (2) | lop (2) | ae! 


Equation (2.41) shows that the output noise spectrum 
acts according to three factors (given that I(z) is small). 
First, the output noise spectrum depends on the input noise 
spectrum. Second, if the signal-to-noise density ratio at 


the reference input is low, the output noise will be low; 
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that is, the smaller the signal component feeding into the 
reference input, the better the cancellation of the noise. 
This is to be expected and was already shown by Equation 
(2.35). The third factor implies that if the signal-to-noise 
density ratio in the primary input (the desired response of 
the adaptive filter) is low, the filter will be trained most 
effectively to cancel the noise rather than the signal and 
therefore the output noise will be low. 

As an illustration of the level of performance attainable 
in practical situations consider the following example. An 
adaptive noise cancelling system is designed to pass a 
plane-wave signal received in the main beam of an antenna 
array and to reject strong interference in the near field or 
in a minor lobe of the array. Assume that the signal and 
interference power spectra are overlapping and that the 
interference power density is twenty times greater than the 
Signal power density at the individual array elements. Then 
the signal-to-noise ratio at the reference input 0 pet is 
1/20. Assume also that because of array gain the signal power 
equals the interference power at the array output which forms 
the primary input to the adaptive system. The signal-to-noise 
ratio at the primary input is p_.. = 1. After convergence 


ovens 
the signal-to-noise ratio at the system output will be 


If signal distortion is defined as 
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BiGzZ) — 0... (z) (2.42) 


then the maximum signal distortion will be 


Diz) = (i720) 71 = 5 percent 


The adaptive cancelling improves the signal-to-noise 
ratio twentyfold while introducing only 5 percent distortion. 
Additionally, the adaptive filter will provide the same per- 
formance when the input conditions change and a new set of 


convergent weights have been obtained. 


D. NOISE CANCELLING APPLICATIONS 

This section describes several practical applications 
which demonstrate the applied concepts of adaptive noise 
cancelling. These applications include cancellation of noise 
in speech signals, cancellation of antenna sidelobe inter- 
ference, cancellation of 60-Hz interference and cancellation 
of either periodic or broadband interference when no reference 
1s available. 

A prime example of noise contaminated speech is that of a 
pilot communicating by radio from the cockpit of an aircraft 
where a high level of engine noise is interfering with the 
pilot's voice. The noise contains, among other components, 
strong periodic mixtures that occupy the same frequency band 
as speech. These components cannot be "low filtered" or 


"high filtered" out of the speech pattern and are picked up by 
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the microphone into which the pilot speaks, severely inter- 
fering with the intelligibility of the radio transmission. 
It 1S impractical to process the transmission with a conven- 
tional fixed filter because the frequency and intensity of 
the noise components vary with engine speed and load and even 
the position of the pilot's microphone. By placing a second 
microphone at a Suitable location in the cockpit, a sample 
of the ambient noise which is free of the pilots speech can 
be obtained. This sample can be filtered and subtracted from 
the transmission, significantly reducing the interference. 
Widrow et al., [Ref. 4] demonstrated the feasibility of 
cancelling noise in speech signals by conducting a number of 
experiments which simulated the cockpit noise problem. Figure 
2.5 shows the system used for cancelling the noise in the 
cockpit noise simulations. A voice input from a room with 
strong acoustic interference was used as the primary input. 
A second microphone was placed in the room away from the 
speaker and this was used as the reference input. The output 
of the noise cancelling was then monitored by a remote listener. 
The canceller included an adaptive filter with 16 weights 
whose values were digitally controlled by a computer. A 
typical experiment used an audio frequency triangular wave 
containing many harmonics as interference. Because of multi- 
path effects the amplitude and phase of the interference 
varied from point to point in the room. The noise cancelling 
system was able to reduce the output power of the interference, 


which otherwise made the speech unintelligible, by 20 to 25 
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dB, rendering the interference barely perceptible to the 
remote listener. No noticeable distortion was introduced 
into the speech pattern. Convergence times were on the order 
of a few seconds and the system was readily able to readapt 
when the position of one or both microphones was changed or 
when the frequency of the interference was varied over the 


range of 100 to 2000 Hz. 


E. CANCELLING ANTENNA SIDELOBE INTERFERENCE 

Another type of noise cancelling is that of eliminating 
strong unwanted signals which are incident on the sidelobes 
of an antenna array. These interferences can severely retard 
the reception of weaker signals on the main beam. The 
conventional method of reducing this type of interference by 
adaptive beamforming is often complex and expensive to imple- 
ment. When the number of spatially discrete interference 
sources is small, adaptive noise cancelling can provide a 
Simpler and less expensive method of coping with this problem. 
Consider an array pattern with signal strengths and directions 
as shown in Figure 2.6. The array consists of a circular 
pattern of 16 equally spaced omnidirectional elements. The 
outputs of the elements are delayed and summed to form a main 
beam steered at a relative angle of 0 degrees. A simulated 
"white" signal consisting of uncorrelated samples of unit 
power is assumed to be incident on the beam. Simulated inter- 
ference with the same bandwidth and with a power of 100 is 


incident on the main beam at a realtive angle of 58 degrees. 
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The entire array is then connected to the adaptive noise 
cancelling system shown in Figure 2.7. In this case the 
output of the beamformer serves as the canceller's primary 
input, and the output of one of the elements (#4) is arbi- 
trarily chosen as the reference input. The adaptive can- 
celler uses 14 weights. 

A number of experiments performed in [Ref. 4] show that 
the Signal-to-noise ratio at the system output was found 
after convergence to be +20 dB. The signal-to-noise ratio 
at the single array element was -20 dB. This result bears out 
the equation shown in the Wiener solution (2.35), that the 
Signal-to-noise ratio at the system output would be the 
reciprocal of the ratio at the reference input, which is 


derived from a single element. 


F. CANCELLING 60-HZ INTERFERENCE IN ELECTROCARDIOGRAPHY 

A practical example of cancelling 60-Hz interference is 
found in electrocardiography. A major problem which exists 
in the recording of electrocardiograms (ECG's) is the ap- 
pearance of unwanted 60-Hz interference in the output. Vari- 
Ous methods have been utilized to help cancel the 60-Hz 
interference, including more effective grounding techniques 
and the use of twisted cabling. Another method capable of 
reducing 60-Hz ECG interference is adaptive noise cancelling 
via a system such as that shown in Figure 2.8. 

The primary input is taken from the ECG preamplifier and 


the 60-Hz reference is taken from a wall outlet. The adaptive 
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filter is simple, containing only two variable weights, one 
applied to the reference input directly and the other toa 
version of it shifted in phase by 90 degrees. The two 
weighted versions of the reference are summed to form the 
filter's output, which is then subtracted from the primary 
input. A valuable advantage in the use of an adaptive filter 
rather than a fixed notch filter at 60-Hz, is that the varia- 
ble weights allow the 60-Hz interference to change in both 
magnitude and phase and still realize effective cancellation. 
G. CANCELLING PERIODIC INTERFERENCE WITH NO EXTERNAL 

REFERENCE 

In many cases where a broadband signal is corrupted by 
periodic interference there is no external reference input 
which is free of the Signal. If a fixed delay is inserted 
in a reference input drawn directly from the primary input, 
as shown in Figure 2.9, the periodic interference can, in 
many cases, be cancelled. A key point is that the delay must 
be chosen to be of sufficient length to cause the broadband 
Signal components in the reference input to become decorre- 
lated from those in the primary input. Because of their 
periodic nature, the interference components will remain 
correlated with each other. 

By taking the output from the LMS output instead of the 
difference signal the same system can be used to separate 
broadband interference from a periodic signal as shown in 


Figure 2.10. Such a filter is often called an adaptive 
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self-tuning filter. Additionally, applications of this filter 
are utilized in the adaptive line enhancer, a system used for 
detection of a low level signal imbeded in noise. The trans- 
fer function of this filter is the digital Fourier transform 
of the impulse response. Its magnitude at the frequency 

of the interference is very nearly one, the value required 


meeeperrect cancellation. 


H. THE ADAPTIVE NOISE CANCELLER AS A NOTCH FILTER 

One of the primary considerations for considering con- 
straints on adaptive filtering systems stems from the filter's 
use as a notch filter. After an analysis of the adaptive 
system as a notch filter it was felt that specific constraints 
could be used to 

a) shape the frequency characteristics 
b) produce a faster convergence. 

Before considering the constraining equations and their 
results it is first helpful to analyze the adaptive noise 
canceller in its notch filter mode. Figure 2.11 depicts a 
Single frequency adaptive noise canceller with two adaptive 
weights. Analytical and experiemntal results show that if 
more than one frequency is present in the reference input 
then a notch for each will be formed. The primary input is 
assumed to be any type of signal--stochastic, deterministic, 
periodic, transient, or any combination of these. The 
reference input is assumed to be a pure cosine wave 


Ss cos (wt +¢). The primary and reference inputs are sampled 
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at the frequency of pee 2(Pi)/T rad/sec. The reference 
input is sampled directly yielding X1(j), and after under- 
going a 90 degree phase shift, also produces X2(j). Assume 
Synchronous sampling. 

A transfer function for the noise canceller of Figure 2.8-1l 
can be obtained by analyzing signal propagation from the 
primary input to the system output. 

The weights are updated in accordance with the LMS 


algorithm yielding 


W1(j+1) = W1(5) + 2yne (3) X1(35) 
(2.43) 
W2(j+1) = W2(5) + 2ype(5) X2(35) 
The sampled reference inputs are 
ri) = Cc cos (wi (j)T + 9) 
(2.44) 
X2(j) = C Sin(w,(j)T + ¢) 


Using signal flow diagram techniques and considering that the 


error Signal at time j = k is 
e(j) = 6(j-k) (2.45) 
the filter's impulse response at k = 0 can be given as 


y(j) = 2uC*u(j-1) cos (wy (3) 7) (2.46) 
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where u(j) is the discrete unit step function. The transfer 


function of this path is 


5 Zz -cosw)T) 


G(z) = 2uCc a. CO uf 
Zz ~2zcosu pT +1 
2 
ZuC.(Z.ces WoT - lL) 
SS a (2.47) 
Ze. cos woT + ] 


This transfer function can be expressed in terms of the 
radian sampling frequency ow = 2(Pi)/T as 
2uC*[z cos (2nwu_-) - 1) 


eg a (2.48) 


Zz -22z cos (2mw)w_) + 1 


When the feedback loop from the adaptive filter output to 
the difference signal is formed, the transfer function H(z) 
from primary input to noise canceller output can be written 


as 


2° -22 cos (27u,w_) + 1 
HOZ) 0 0 ae ee (2.49) 


Zo -2(1 -uC*)z cos (2mw pw.) +1 -2uC* 


This transfer function has the property of a notch filter at 


the reference frequency w The zeros of the transfer func- 


0° 
tion are located in the z-plane at 


wt) 2S) 


zZ = exp(tj2mw, - 


and are exactly on the unit circle at angles of t2mw pw 


radians. The poles are inside the unit circle at a radial 
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distance (1-2uc7) 172 


from the origin. For slow adaptation 
(small values of te} the angles of the poles are almost 
identical to the zeros. Since the zeros lie on the unit 
Circle, the depth of the notch in the transfer function is 


infinite at frequency » = »w The sharpness of the notch is 


0° 
determined by the closeness of the poles to the zeros. Corres- 
ponding poles and zeros are separated by a distance approxi- 


mately equal to eae The notch bandwidth is given by 


BW = UC*w -/n 22.51) 


and the Q of the notch is determined by the ratio of the 


Center frequency to the bandwidth. 
QqQ = —_——— (Zee) 


The single frequency noise canceller is, therefore, equiva- 
lent to a stable notch filter when the reference is a pure 
cosine wave. The depth of the null is generally superior 
to that of a fixed digital or analog filter because the 
adaptive process maintains the null exactly at the reference 


iPiaecguency . 
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Pets lose CONS RALNED ADAPTIVE FIR FILTER 


In this chapter the concept of the constrained adaptive 
filter is introduced. A literature search has indicated that 
very little research has been done in the area of constrained 
adaptive filters. Frost [{Ref. 8] presents a constrained LMS 
algorithm which is capable of adjusting an array of sensors 
in real time to respond to a signal coming from a desired 
direction while discriminating against noises coming from 
other directions. A set of linear constraints on the weights 
Maintains a chosen frequency characteristic for the array in 
the direction of interest. 

In this chapter three constraint conditions are presented. 
The first two involve a constraint on the angle between zero 
locations, so that this angle remains constant while the zero 
location changes adaptively. The constraint essentially 
Maintains the pattern of the zeros while their location is 
shifted. The first approach is a direct implementation in 
which one of the weights is changed adaptively using the LMS 
algorithm and the others are slaved by the fixed angle formula 
to the adaptively adjusted weight. The second implementation 
involves a cascaded version of the foregoing master-slave 
Sencept. The third constraint considered is a linear con- 
straint on the weights. The approach presented uses a La- 
grangian multiplier formulation to augment the cost function 


in which the basic LMS adaptive algorithm is applied. 
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A. FIXED ANGLE CONSTRAINT--DIRECT IMPLEMENTATION 

Consider a transversal filter using the LMS algorithm 
with the constraint that the angular separation between the 
filter zeros in the z-plane is to be a constant as set by 
design requirements. As an example, consider a fourth order 


FIR filter with B = 86 where 8, and 9. are the angles 


A Ne 1 2 

of the zeros. The value of B and the magnitude of the zeros 
1s to be kept constant while the angles 95 and 85 are to be 
adjusted adaptively. This specification serves to maintain 
the shape of the filter characteristics in the frequency 
domain. From the direct implementation of the transversal 


filter shown in Figure 3.1 the system transfer function is 


given by 


agi 


H(z 7) (Se) 


it 
r- 
4 
= 
N 
{ 
at 
= 
N 
ae 
= 
N 
a 
= 
N 


= (z )(z + Wz + Wz tei coats W) (3.2) 


which can be factored into the form 


(2° -2r.,cosé z+4%) (20-24 cos0,z +x5) 


al it 2 


D(z) ot 





(S53) 


where ry and I. represent the radius of the zero placement 
in the z-plane and 95 and 9, are their respective angular 
displacement from the axis. Carrying out the indicated 


mibcr1plication of the numerator of (3.3) yields 
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CONSTRAINT 
ALGORITHM 
x(k-1) 


LMS d(k ) 


Begs 3.1, Transversal Adaptive Filter - Direct 
Implementation 
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= 4 3 2 2,2 
maiz) = 2z (2x, cos6,+2r,cos8.)z + (rj +4r,r.,cos8,cosé,tr,)z 
Z Ye ae 

- (2r)r,cos6, +2r,r,cosé,)z + rit (3 4} 
Setting these terms equal to the weight values of (3.2) 
yields 
Wo = J] (335.5) 
Wy = -2(r, cos6, + ©, cos 9.) (3 26) 
W. = (4° +4 cos¢, cos. +r°) eon 
2 itn lo 92 =D 
W, = (-2r)4r,) (r, cos 6. +X, cos 9,) (300 
_ ee 
ae 1% (259) 
The adaptive LMS algorithm must now be constrained so that 
the angle B is given by 
Bee = 6 (2310) 
Hl 2 
For simplicity let 
els S R (oer) 
fmeem (3.10) it follows that 
9, (k+l) = Bt 8. (k+1) sal 2) 


Using (3.11) and (3.12) Equation (3.6) can now be written as 
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W, (k+l) ——e-7hicesth +o, (ktil)) + cos (9. (k+1)) Come a) 


2 
Using the trigonometric identity 
cos(A+B) = cos AcosB - sinAsinB 


Equation (3.13) can be written as 


W, (k+1) = -2R(cos Bcos (6, (k+1)) -sin Bsin(6,(k+1)) 
+ cos (8, (k+l) ) oe ) 
Combining terms, and defining the constants 
K3 = ~-2R(cos(B) + 1) Spied) 
K4 = =-2R(sin(B) ) (GS 2.6) 
Equation (3.13) can finally be written as 
W, (k+l) = K3 cos (8,(k+1)) - K4 sin (6, (k+1)) (3547) 


This transcendental equation can be solved iteratively for 
8, (k+1) using the value for W, (k+1) obtained from the LMS 


algorithm 
W, (k+1) = W, (x) + 2uX(k-1) « (k) (63 eles) 


Now that 6.(k+1) is known, Equation (3.7) can be solved for 


2 
W, (k+l), where 
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2 
W, (k+l) = 2R + aR? cos (6, (k+1) +B) cos (8, (k+1)) (3.19) 


Meeeqnazing that Equation (3.8) is proportional to Equation 


(3.6), the fourth algorithm iteration can be written as 


W,(k+1) = RoW, (k+1) (3.20) 


From (3.9) Wa is a constant value 


W,(k+1) = R4 ers 
The four equations (3.18) through (3.21) comprise an 
adaptive iterative algorithm that produces the direct analyti- 
cal realization of Figure 3.1. It is seen that Ws is changed 
adaptively and Wo and W, are slaved to Whe and Wo and Wy are 
constants. This approach yields two pairs of zeros which are 
located at radius R from the origin of the z plane and precisely 


B degrees apart. Figure 3.2 depicts the resulting z-plane 


diagram. 


B. FIXED ANGLE CONSTRAINT--CASCADED IMPLEMENTATION 
A cascaded implementation of the fourth order transversal 
filter is shown in Figure 3.3. The transfer functions result- 


ing from Figure 3.3 are given by 


2° - 2x, cos 8,2 ” mG 
H, (2) aa oe Gorn 2) 


Z 


for the first section, and for the second section 
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Fig. 3.2. 2Z-Plane Diagram of Zeros Resulting from 


Transversal Filter 
IV indicates four poles located at the 


Oneal e(als@ 
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Fig. 3.3. Cascaded Transversal Filter with LMS 
Adaptation of Wi: Master-Slave Concept 
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2° - 2X. cos 8,2 +15 
H. (Zz) = a (3223) 


Z 
The weight values from these two equations are 


W = -=-2 r.cos 8 


ii 1 ut 
Wo = oa 
W, = --2 r. cos 85 
Wy = % 


Once again the algorithm producing a set of converging 


weights begins with the LMS adaptive equation 
W, (k+l) = W, (k) + 2uX(k-1) « (k) 35524) 
The second weight is a constant proportional to 
2 


W. (k+l) = R G52 5) 


From (3.22) the value for W., can be written as 


IL 
W, (k+l) = - 2 rj, cos 8, (k+l) (34:26) 
Solvang for 6, (k+l) 
“W. (k+1) 
cosé, (k+1) = oa (Sac) 


where the value of W, (k+1) is known from (3.24) and 


68 





Therefore, 
8, (k+1) = Arccos ine (——s,—_) (3723) 
Now the third weight can be written in the form of 
W, (k+l) = -2 Rcos6. (k+l) (3.29) 
where 
8. (k+1) = B- 8, (k+l) (3% 30) 


with B a given constant set by the user and 8, (k+l) known 


From (3.27). The fourth weight is proportional to 


W,(ktl) = R* (3.31) 
@iemrour equations (3.24), (3.25) and (3.29), and (3.31) 
comprise an adaptive iterative algorithm that produces the 
cascaded realization of Figure 3.3. As in the direct reali- 
zation, the solution of Equations (3.22) and (3.23) using the 
adaptive weight solutions produces two pairs of zeros which 
are located at radius rj = 5 = R from the origin of the 


z-plane and precisely B degrees apart. 


C. LINEARLY CONSTRAINED WEIGHTS--LAGRANGE MULTIPLIER APPROACH 


Consider now the LMS adaptive equations modified for a 


different type of constraint, namely a linear constraint on 
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the sum of the weight vector. That is, the constraint that 
W, (x) + W. (k) en ag W (kK) = K fo 2) 


where K is some user defined constant. This constraint func- 
tion is adjoined to the square of the error equation by the 
method of Lagrange multipliers [Ref. 9]. Recalling from 
Pem@eation (1.5) that 

Ak 
) 


Sls = Cl hs) SU) 6 2S bas), 


the adjoined and modified least mean squared error function 


can be written as the cost function equation 


S(k) = [d(k)-w(k)7X(k)]? = a (k) (w(k)7u -K] (3.33) 


where 


ee it... 1)” 
and A(k) is the scalar Lagrange multiplier. 

In order to minimize the mean squared error under the 
given constraint consider the gradient of (3.33) with respect 


to the weights and to the Lagrangian multipliers. 


VIT(k) 1, -~2 [d(k) -W7 (k) X(k)]X(k) - A(k)u (3.34) 


and 
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Vid(kK)}, = -{W(k)“u - KI (3.35) 


The method of steepest descent can be described by the two 


relationships 

W(k+1l) = W(k) + See (250) 
and 

wikel) = ACK) + kK, V{S(k)], G3). 67) 


Substituing Equations (3.34) and (3.35) into Equations (3.36) 
and (3.37) yields the final weight and Lagrange parameter 


iteration algorithms using the LMS approximation 


W(k+l) = W(k) - 2k, e(k) X(k) Ses Ghee) (350) 

and 

\(k+1) = A(k) - ky (W(k)7u - KI (3.39) 
where 

W(k) = the weight vector before adjustment 

W(k+l) = the weight vector after adjustment 

K = the linear constraint value 

k, +k, = the scalar constants (ky ,Ko < 0) 
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A(k) = the Lagrange parameter before adjustment 


A(k+1) = the Lagrange parameter after adjustment. 


The resulting constrained adaptive filter is shown in Figure 


3.4. Experimental results are presented in the next chapter. 
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Fig. 3.4. The Adaptive LMS Filter with Linear Constraint 
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IV. SIMULATIONS AND RESULTS 


Using the Hewlett-Packard 85 (HP-85) microcomputer 
several computer simulations are performed to demonstrate 
both the unconstrained and the constrained LMS adaptive 
systems discussed in Chapters II and III. The unconstrained 
adaptive system utilized for computer simulation is the 
system depicted in Figure 2.1. The unconstrained results 
are obtained first with nine adaptive weights and then with 
only four adaptive weights. The noise input consists of 
either the sum of two sinewaves or zero mean white noise 
added to a sinewave. In all unconstrained cases the 
reference input (the desired waveform) is a sinewave. The 
constrained adaptive systems are simulated for the linear 
constraint using the Lagrangian multiplier and for the 
angular constraint (separation of zeros in the z-plane 


plot) in both cascaded and direct implementations. 


A. THE UNCONSTRAINED ADAPTIVE NOISE CANCELLER 

The adaptive noise canceller shown in Figure 2.1 is 
computer simulated using nine weights, each adapted by the 
LMS algorithm. The desired signal is a 10 Hz sinewave 
sampled at f = 128 Hz. The noise input is the sum of two 
equal sinewaves with frequency f = 10 Hz and frequency 
f = 35 Hz. The number of samples is N = 128 and the 


adaptation constant is y = K = 0.1. The system output 
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converges after a short learning period to the desired 
Signal. Figures 4.1 through 4.3 depict these results. 

A sample of three of the Nine weights (Wo. Was Wa) LS 
shown in Figures 4.4 through 4.6 to illustrate the steady 
state solution of the adaptive weights. After just N = 24 
all weights have settled to within 0.1% of their final 
value. Figure 4.7 1s the system error. It should be noted 
that a faster sampling frequency of £f = 256 reduces this 
error even further. 

The next Simulation is Similar to the first with the 


exception that this system uses only four weights instead 


of the nine weights used previously. Again the noise input 


is the sum of two equal sinewaves (f 4 Hz, £ = 20 Hz) and 


the reference or desired waveform iS a sinewave at f = 4 Hz. 


The adaptation constant is Kl = 0.1, N 128 and the 


128 Hz. Figures 4.8 


Sampling frequency for the inputs is f 
and 4.9 show the filter input and the reference signal. It 
is evident from the system output shown in Figure 4.10 that 
this system does not track as well as the previous system 
with nine weights. Plotting weight #1 of the system through 
time, as shown in Figure 4.11, illustrates the oscillatory 
nature of the weight values. 

The same unconstrained simulation is repeated with the 
noise input taken as zero mean uncorrelated noise (generated 
by the HP-85 random number generator) summed with a sinewave 


of frequency £ = 10 Hz. Figure 4.12 shows the noise input. 
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The desired signal is the sinewave shown in Figure 4.13. 
Nine adaptive weights are utilized. The system sampling 
frequency is f = 256 Hz and the adaptation constant is 

Kl = 0.1. The signal-to-noise ratio of the input signal 

is 1.64 dB. Figure 4.14 shows the system output while 
Figure 4.15 illustrates the system error. Figure 4.16 shows 


one of the system weights plotted through time. 


B. CONSTRAINED ADAPTIVE SYSTEM SIMULATIONS 

The simulation of the angularly constrained adaptive 
systems shown in Figures 3.1 and 3.3 produces nearly identical 
results. Computationally, the iterative solution required 
for the direct implementation is much slower than that of 
the meeeaded system. Since both system outputs are virtually 
identical, those of the faster cascaded system are shown. 

All of the linearly constrained adaptive results utilize 
nine adaptive weights and all of the angularly constrained 
systems utilize four weights. 

Using the cascaded adaptive system shown in Figure 3.3 
Simulation results are obtained for a noise input shown in 
Figure 4.17 of two sinewaves of frequencies f = 4 Hz and 
f = 20 Hz. The sampling frequency f = 128 Hz. The 
desired signal shown in Figure 4.18 is a sinewave of 
frequency f = 4Hz. The adaptation constant is Kl = 0.l. 

The zeros are desired at a radius of 1 and with a separation 


of 12 degrees. The system output tracks the desired signal 
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as shown in Figure 4.19 and places the system zeros with 
the exact requested degree of separation as shown in 
Figure 4.20. Zero 1 is at radius 1 and an angle of 63.39 
degrees. Zero 2 1s at a radius of 1 and an angle of 51.39 
degrees. The other two zeros are the conjugate pairs of 
the first two. 

The adapted weight number 1 is plotted through time in 
Figure 4.21. The same oscillating pattern is shown here as 
is depicted in Figure 4.11. It is felt that, as in the case 
of Figure 4.11, using a greater number of weights would 
eliminate the noisy weight pattern and accordingly produce 
a cleaner output. 

Suppose now that the user wishes to drastically alter 
the frequency characteristics. The input to the system 
remains the same (as Shown in Figure 4.17) however now the 
desired angle of separation of the zeros is 130 degrees 
instead of the original 12 degrees. System output is shown 
in Figure 4.22, the reference Signal is shown in Figure 4.23, 
and the z-plane zero location is shown in Figure 4.24. 

Zero number 1 is at a radius of 1 and at an angle of 158.55 
degrees and zero number 2 is at a radius of 1 and an angle 
of 28.55 degrees, providing the prescribed 130 degrees of 
Separation. The remaining two zeros form the conjugate pair. 
The sampling frequency is changed to f = 256 Hz and the 


adaptation constant Kl is changed to 0.05. 
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The same simulations are performed for two sSinusoids of 
Frequencies f = 10 Hz and f = 35 Hz. A Sampling frequency 
of £ = 256 Hz is used with an adaptation constant Kl = 0.1. 
Figures 4.25 through 4.28 depict the system input (the sum 
of the two sinusoids), the filter output, the desired signal, 
and the z-plane zero locations for a desired separation of 
20 degrees. Zero 1 is located at a radius of 1 and an angle 
of 50.2 degrees. Zero 2 is located at a radius of 1 and an 
angle of 70.2 degrees. The remaining two zeros are the 
conjugate pairs. Figures 4.29 and 4.30 show the system out- 
put and z-plane zero location for the same system input and 
same reference but with a desired zero Separation of 40 
degrees. Zero 1 is located at a radius of 1 and an angle 
of 45.43 degrees and zero 2 is located at a radius of 1 and 
an angle of 85.43 degrees. The remaining two zeros are 
conjugate pairs. 

C. LINEARLY CONSTRAINED ADAPTIVE FILTER SIMULATIONS 

USING THE LAGRANGE MULTIPLIER TECHNIQUE 

The linearly constrained LMS adaptive filter shown in 
Figure 3.4 is simulated for various values of K, the linear 
constraint value. Two types of system inputs are used-- 
random "white" noise and a Sinusoidal signal of frequency 
4 Hz. In each case the desired signal is a damped exponen- 
tial with a final value of 0.2. 

Figure 4.31 depicts the filter output superimposed upon 


the desired output of a damped exponential of final value 
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Fig. 4.25.: System Input - Four Adaptive Weights 
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Fig. 4.26. System Output - Four Adaptive Weights 
Desired Zero Separation = 20 Degrees 
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Fig. 4.27. Reference Signal 
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4.28. 2Z-Plane Zero Locations - Four Adaptive 
Weights 
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Fig. 4.29.. System Output - Four Adaptive Weights 
Desired Zero Separation = 40 Degrees 


107 







j =— 


7 
a 


sec in’ USTUO RST Iaa 
evorlTtsse ere 


7 


td 
> 
i 


a 
7 






a 


, 

i. 

@ ; | 
——tin y 

I 

SugivO mpzevs ., 28.6 


Lod 


2b nO ash bre or 
ih ad 





RADIUS=1 | 
BPESITRED ANGLE “ser=4e 


Fig. 4.30. 2Z=-Plane Zero Location 
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Fig. 4.31. System Output and Desired Signal 
Linear Constraint K = 0 
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0.2. The linear constraint K is arbitrarily chosen to be 
zero. The adaptation constant is Kl = 1 E-6. The system 
error is shown in Figure 4.32. The sum of the weights plotted 
through time is shown in Figure 4.33. The final value of 
the Lagrange multiplier, that is A(N) at time N = 64, is 
found to be -1.022 E-4. The final value of the sum of 
weights is found to be 0.059 compared with a requested value 
Saeco ro. 

The same system is again Simulated for a linear con- 
Straint of K = -0.2. Figure 4.34 shows that the sinusoidal 
input converges to the desired damped exponential of final 
value 0.2. Interestingly, the response converges and then 
begins to build an oscillation. This type of response in 
adaptive systems has been observed elsewhere and may be due 
to an arithmetic precision problem [Ref. 10]. Figure 4.35 
shows the system error and Figure 4.36 shows the sum of 
weights graph plotted through time. The final value of the 
sum of weights is found to be -0.2427 compared with the 
desired value of -0.2. 

The simulation is again performed, now with a "white" 
noise input and the same damped exponential with final value 
of 0.2 as the desired signal. In the first of the three 
noise input simulations the sum of weights constraint K is 
set to -0.1; the second simulation has K = 0.2; the final 
simulation uses K = 0.5. Figures 4.37 through 4.42 depict 


the simulation results. In Figure 4.42 the sum of the 
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Fig. 4.33. Sum of Weights Plotted Through N = 128 
Samples. Constraint K = 0 
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Fig. 4.34. System Output and Desired Signal 
K = -.2 
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Fig. 4.36. Sum of Weights Plotted Through N = 128 
Samples. Linear Constraint K = -.2 
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Fig. 4.38. Sum of Weights Plotted Through N = 64 
Samples. Constraint K = -0.1 
Nine Adaptive Weights 
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Fig. 4.40. Sum of Weights Plotted for N = 128 Samples 
Nine Adaptive Weights. K = .2 
Kl = .00001 K2 = .00008 
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Fig. 4.41. System Output Superimposed over the 
Desired Signal of a Damped Exponential 
K = .5 
Kl = .00001 K2)——. 00009 
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Dee ONCLUSIONS 

This thesis has investigated adaptive noise cancellation 
techniques in detail with the objective of implementing 
constrained adaptive filters. Two basic types of constrained 
adaptive filters are presented--the fixed angular separation 
(direct and cascaded implementations) and the adaptive 
Lagrangian multiplier linear constraint approach. All three 
methods produce the desired output within a very close 
tolerance for the examples tested. As expected, the direct 
implementation takes longer to converge than the cascade 
approach. The results are promising, however, several key 
questions remain to be investigated. In particular the 
convergence properties of these constrained adaptive 
algorithms need to be studied in detail. In the case of 
the Lagrangian multiplier approach, if there is an optimal 
steady-state solution, the modified LMS algorithm should 
find a minimum since it is a simplified gradient technique. 
However the LMS minimum may not be global. For the case of 
the angular constraint process, the stability of the 
adaptive algorithm and its convergence also need to be 
examined, although the master-slave would seem to contain 


inherent stability. 
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Finally, the concepts of constrained adaptive algorithms 
need to be examined with more sophisticated examples using 


a larger number of weights. 
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